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1. A method of tracking facial features in a video sequence, said method 
comprising the steps of: 

(a) receiving facia! features for tracking in a first frame of said video sequence; 

(b) spatiotemporally segmenting said video sequence to provide a sequence of 
associated two-dimensional segments, a first segment in said sequence of associated two- 
dimensional segments including said facial features for tracking; 

(c) identifying candidate facial features in at least a second two-dimensional 
segment in said sequence of associated segments; and 

(d) verifying which of said candidate facial features correspond with said facial 
features for tracking. 

2. A method as claimed in claim I comprising the further step of: 

(e) recovering lost facial features by using known geometric relations between 
facial features. 

3. A method as claimed in claim 1 wherein step (c) comprises the sub-steps of: 
(ci) forming a sub-image including said two-dimensional segment in said 

sequence of associated segments; 

(cii) normalising the size of said sub-image; and 

(ciii) identifying candidate facial features in said normalised sub-image. 

4. A method as claimed in 1 wherein step (d) measures the correspondence 
between said candidate facial features and said facial features for tracking. 
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5. A method as claimed in claim 4 wherein step (d) comprises determining 
whether said candidate facial features are within a region of said facial features for 
tracking in a previous frame. 

6. A method as claimed in claim 5 wherein step (d) further comprises 
determining whether said candidate facial features within each of said regions that are 
similar in shape to said facial features for tracking in said previous frame. 

7. A method of tracking facial features in a sequence of associated two- 
dimensional segments, a first segment in said sequence of associated two-dimensional 
segments including facial features for tracking, said method comprising the steps of: 

(a) identifying candidate facial features in at least a second two-dimensional 
segment in said sequence of associated segments; 

(b) verifying which of said candidate facial features correspond with said facial 
features for tracking; and 

(c) recovering lost facial features by using known geometric relations between 
facial features. 

8. A method as claimed in claim 7 wherein step (a) comprises the further sub- 
steps of: 

(ai) forming a sub-image including said two-dimensional segment in said 
sequence of associated segments; 

(aii) normalising the size of said sub-image; and 

(aiii) identifying candidate facial features in said normalised sub-image. 
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9. A method as claimed in claim 7 wherein step (b) measures the 
correspondence between said candidate facial features and said facial features for 
tracking. 

5 

10. A method as claimed in claim 9 wherein sub-step (b) comprises 
determining whether said candidate facial features are within a region of said facial 
features for tracking in a previous frame. 

10 11. A method as claimed in claim 1 0 wherein sub-step (b) further comprises 

determining whether said candidate facial features within each of said regions that are 
similar in shape to said facial features for tracking in said previous frame. 

12. An apparatus for tracking facial features in a video sequence, said 
15 apparatus comprising: 

means for receiving facial features for tracking in a first frame of said video 
sequence; 

means for spatiolemporally segmenting said video sequence to provide a 
sequence of associated two-dimensional segments, a first segment in said sequence of 
20 associated two-dimensional segments including said facial features for tracking; 

means for identifying candidate facial features in at least a second two- 
dimensional segment in said sequence of associated segments; and 

means for verifying which of said candidate facial features correspond with said 
facial features for tracking. 

25 
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13. An apparatus as claimed in claim 1 2 further comprising: 

means for recovering lost facial features by using known geometric relations 
between facial features. 

14. An apparatus as claimed in claim 12 wherein said means for identifying 
comprises: 

means for forming a sub-image including said two-dimensional segment in said 
sequence of associated segments; 

means for normalising the size of said sub-image; and 

means for identifying candidate facial features in said normalised sub-image. 

15. An apparatus for tracking facial features in a sequence of associated 
two-dimensional segments, a first segment in said sequence of associated two- 
dimensional segments including facial features for tracking, said apparatus comprising: 

means for identifying candidate facial features in at least a second two- 
dimensional segment in said sequence of associated segments; 

means for verifying which of said candidate facial features correspond with said 
facial features for tracking; and 

means for recovering lost facial features by using known geometric relations 
between facial features. 

16. An apparatus as claimed in claim 1 5 wherein said means for identifying 
comprises: 

means for forming a sub-image including said two-dimensional segment in said 
sequence of associated segments; 
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means for normalising the size of said sub-image; and 

means for identifying candidate facial features in said normalised sub-image. 



1 7. A program stored on a memory medium for tracking facial features in a 
5 video sequence, said program comprising: 

code for receiving facial features for tracking in a first frame of said video 
sequence; 

code for spatiotemporally segmenting said video sequence to provide a sequence 
of associated two-dimensional segments, a first segment in said sequence of associated 
10 two-dimensional segments including said facial features for tracking; 

code for identifying candidate facial features in at least a second two- 
dimensional segment in said sequence of associated segments; and 

code for verifying which of said candidate facial features correspond with said 
facial features for tracking. 

15 

18. A program as claimed in claim 1 7 further comprising: 

code for recovering lost facial features by using known geometric relations 
between facial features. 



20 19. A program as claimed in claim 17 wherein said code for identifying 

comprises: 

code for forming a sub-image including said two-dimensional segment in said 
sequence of associated segments; 

code for normalising the size of said sub-image; and 
25 code for identifying candidate facial features in said normalised sub-image. 
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20. A program stored on a memory medium for tracking facial features in a 
sequence of associated two-dimensional segments, a first segment in said sequence of 
associated two-dimensional segments including facial features for tracking, said program 
comprising: 

code for identifying candidate facial features in at least a second two- 
dimensional segment in said sequence of associated segments; 

code for verifying which of said candidate facial features correspond with said 
facial features for tracking; and 

code for recovering lost facial features by using known geometric relations 
beiween facial features. 

21. A program as claimed in claim 20 wherein said code for identifying 
comprises: 

code for forming a sub-image including said two-dimensional segment in said 
sequence of associated segments; 

code for normalising the size of said sub-image; and 

code for identifying candidate facial features in said normalised sub-image. 
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